27th AIAA Aerodynamics Measurement and Ground Testing Conference 
28 June - 1 July 2010, Chicago, Illinois 


AIAA 2010-4202 


Comparison of Iterative and Non-Iterative 
Strain— Gage Balance Load Calculation Methods 

N. Ulbrich* 

Jacobs Technology Inc., Moffett Field, California 9f 035-1000 

The accuracy of iterative and non-iterative strain gage balance load calcu- 
lation methods was compared using data from the calibration of a force balance. 
Two iterative and one non-iterative method were investigated. In addition, 
transformations were applied to balance loads in order to process the calibra- 
tion data in both direct read and force balance format. NASA’s regression 
model optimization tool BALFIT was used to generate optimized regression 
models of the calibration data for each of the three load calculation methods. 
This approach made sure that the selected regression models met strict statisti- 
cal quality requirements. The comparison of the standard deviation of the load 
residuals showed that the first iterative method may be applied to data in both 
the direct read and force balance format. The second iterative method, on the 
other hand, implicitly assumes that the primary gage sensitivities of all balance 
gages exist. Therefore, the second iterative method only works if the given bal- 
ance data is processed in force balance format. The calibration data set was 
also processed using the non-iterative method. Standard deviations of the load 
residuals for the three load calculation methods were compared. Overall, the 
standard deviations show very good agreement. The load prediction accuracies 
of the three methods appear to be compatible as long as regression models used 
to analyze the calibration data meet strict statistical quality requirements. Re- 
cent improvements of the regression model optimization tool BALFIT are also 
discussed in the paper. 


Nomenclature 


AF = axial force 

Bi = square matrix that is used in the iteration equation of Iterative Method 2 

B 2 = square matrix that is used in the iteration equation of Iterative Method 2 

Ci = square matrix that is used in the iteration equation of Iterative Method 1 

C 2 = rectangular matrix that is used in the iteration equations of Iterative Method 1 & 2 

Fi = balance load 

F 0 = F-Value of Regression of a regression model 

F = part of matrix G that contains loads 

G = load matrix 

H = part of matrix G that contains absolute value and non-linear terms 

i = dependent variable index -or- summation index 

k = number of regression model coefficients (not counting the intercept term) 

n = number of data points or observations of an experimental data set 

NF = total normal force 

N 1 = normal force component at the forward gage of a balance 

N 2 = normal force component at the aft gage of a balance 

m = number of independent variables 
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= distance between forward and aft normal force gage 
= total pitching moment at the balance moment center 
= distance between forward and aft side force gage 
= total side force 

= side force component at the forward gage of a balance 
= side force component at the aft gage of a balance 
= electrical output of a strain-gage 
= total rolling moment 
= total sum of squares 
= sum of squares due to residuals 
= sum of squares due to regression 
= measured dependent variable or response 
= fitted dependent variable or response 
= total yawing moment 
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= independent variable 
= independent variable 
= delta bridge output vector or matrix 
= dependent variable 

= coefficient used in regression model of strain-gage outputs (iterative methods) 
= intercept term 

= coefficient used in regression model of loads (non-iterative method) 

= intercept term 
= iteration step index 
= standard deviation 


I. Introduction 

Different methods are used in wind tunnel testing in order to compute balance loads from measured 
strain-gage outputs. These methods may be separated into two classes: iterative and non-iterative methods. 
In principle, both classes of methods fit a given set of balance calibration data in the least squares sense and 
use the result of the regression analysis for the prediction of balance loads during a wind tunnel test. The 
two classes of methods, however, differ in the selection of the independent and dependent variables that are 
used for the regression analysis of the balance calibration data. 

Two iterative methods (method 1 and method 2) are currently being used in the wind tunnel testing 
community. These iterative methods fit the gage outputs as a function of the applied calibration loads and 
use an iteration equation in order to predict loads from gage outputs. The non-iterative method, on the 
other hand, directly fits the balance calibration loads as a function of the measured gage outputs. Table 1 
summarizes basic differences between the regression analysis of balance calibration data using iterative and 
non-iterative methods. 

Table 1: Differences between strain-gage balance calibration analysis methods. 



ITERATIVE METHODS 
(METHOD 1 & METHOD 2) 

NON-ITERATIVE METHOD 
(DIRECT-COMPUTE METHOD) 

INDEPENDENT VARIABLE 

LOADS 

GAGE OUTPUTS 

DEPENDENT VARIABLE 

GAGE OUTPUTS 

LOADS 


It is often unclear how the choice of the balance calibration analysis method, i.e., the choice of the 
load calculation method, influences the load prediction accuracy of the balance. In addition, the processing 
of data in direct-read or force/moment balance format may also have an influence on the load prediction 
accuracy. Therefore, it was decided to perform a study that would use a realistic balance calibration data 
set in order find answers to some of these questions. Results of this investigation are discussed in great detail 
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in the present paper. 

During the study three load calculation methods were applied to balance calibration data that was 
given in two load formats. Therefore, a total of six independent regression analysis cases were investigated. 
The three different load calculation methods are explained in some detail in the next section of the paper. 
Then, calibration data used in the present study is described and final results of the load prediction study 
are presented and compared. Some recent improvements of a regression model optimization process are also 
discussed as this optimization process was used to obtain regression models of the calibration data for the 
current study. 


II. Description of Load Calculation Methods 

In general, iterative methods use a two-step approach in order to determine loads from measured gage 
outputs. First, electrical outputs of balance strain-gages are fitted as a function of calibration loads. Then, 
the loads are computed using an iteration equation that is derived from the global regression analysis of the 
strain-gage outputs. The balance loads are the independent variables of the regression analysis problem. 
The electrical outputs of the gages, on the other hand, are the dependent variables. Therefore, the regression 
model of the gage outputs can be expressed as follows: 

Ri = Co(i) + Cl(*) • -Fl + C2 (i) ' F2 + ••• + C,m{i) • Fm + ••• ( 1 ) 

Detailed explanations of iterative methods are given in Refs. [1], [2], and [3]. For the present study 
two iterative methods, i.e., Iterative Method 1 and Iterative Method 2 were selected. Iterative Method 1 is 
described in great detail in Ref. [1]. Iterative Method 2 is explained in Ref. [3]. Figure 1 compares the 
definition and initial guess of the iteration equations that the two iterative methods use (from Ref. [3]). 

The Non-Iterative Method exchanges the independent and dependent variables that Iterative Method 1 
and Iterative Method 2 use. Now, the strain-gage outputs become independent variables and the balance 
loads are the dependent variables for the regression analysis of the balance calibration data. Therefore, the 
balance loads are fitted as a function of the measured strain-gage outputs. 

Fi = r? 0 (i) + + ?? 2 (*) • R 2 + ••• + ■ Rm + •■■ (2) 

The Non Iterative Method has the advantage that it is a one-step method. No iteration is needed in 
order to compute loads from measured strain-gage outputs during a wind tunnel test. An analyst, however, 
must not forget that the method ignores the fact that the balance loads are the “true” independent variables 
of the calibration experiment as loads are “applied” and strain-gage outputs are “measured” during the 
calibration of a balance. Therefore, the success of the Non-Iterative Method depends on the fundamental 
assumption that an exchange of the independent and dependent variables of the calibration data set does 
not negatively influence the mathematical description of the “true” physical behavior of the balance. 

III. Description of Balance Calibration Data 

Machine calibration data of NASA’s MC60D balance was selected for the present study. The balance 
is shown in Fig. 2. It is a force balance that was manufactured by Triumph/Force Measurement Systems in 
2008. Table 2 lists load capacities of the balance in direct read format: 

Table 2: Load capacities of the MC60D balance in direct-read format. 



NF, lbs 

PM, in-lbs 

SF, lbs 

YM, in-lbs 

RM , in-lbs 

AF, lbs 

CAPACITY 

5000 

20000 

2500 

10000 

5000 

700 


The balance calibration data was obtained in Triumph’s balance calibration machine (ABCS). The 
supplied machine calibration data of the balance was already corrected for the weight of the balance shell 
and other calibration fixtures. This characteristic was an advantage for the current study as the fitted 
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balance loads were not influenced by tare load estimate differences that may be caused by small numerical 
differences between the selected load calculation methods. 

In general, it is important to assess the accuracy of a load calculation method at (i) data points that are 
used to develop the regression model of the balance characteristics and at (ii) confirmation points that are 
independent of the regression model. Therefore, it was decided to split the original machine calibration data 
set of the MC60D balance into two subsets. The first subset (subset 1) was used to develop the regression 
models for the three load calculation methods. The second subset (subset 2) contained confirmation points. 
Table 3 below summarizes important characteristics of the balance and the calibration data set that was 
used for the present study. 

Table 3: Balance and calibration data set characteristics of the MC60D balance. 


BALANCE NAME 

MC-60H-2.00D 

BALANCE TYPE 

FORCE BALANCE 

DIAMETER 

2.0 [in] 

GAGE DISTANCE (p, q) 

6.0 [in] 

CALIBRATION DATE 

DECEMBER 2008 

CALIBRATION METHOD 

MACHINE CALIBRATION 

TOTAL NUMBER OF CALIBRATION POINTS 

1906 

NUMBER OF DATA POINTS 

953 

NUMBER OF CONFIRMATION POINTS 

953 

LOAD FORMAT 

TARE CORRECTED LOADS 

GAGE OUTPUT FORMAT 

GAGE OUTPUT DIFFERENCES 


Balance load residuals are defined as the difference between the measured and fitted (predicted) loads. 
Standard deviations of the residuals of both subsets were computed in order to compare the predictive 
capability of the three load calculation methods. In addition, the load capacities listed in Table 2 were 
used to express the standard deviation of the residuals in units of percent whenever data was processed in 
direct-read format. 


IV. Regression Model Optimization Process 

It was decided to use only regression models of the calibration data for the current study that meet 
strict statistical quality requirements. These regression models were identified using a regression model 
optimization process that was implemented in a regression analysis tool called BALFIT (see Refs. [2], [3], 
and [4] for a description of BALFIT’s optimization process). 

BALFIT uses a candidate math model search algorithm during the optimization of the regression model 
of a given set of multivariate experimental data. This optimized regression model is called “recommended 
math model.” It is used for the final regression analysis of the experimental data set. In principle, the 
search algorithm minimizes a search metric by testing only those math models that (i) do not have linear 
or near-linear dependencies between terms and that (ii) satisfy a set of search constraints. The search 
metric, for example, may be the standard deviation of the PRESS residuals of the dependent variable of the 
regression analysis problem (see Ref. [5] for a description of different search metrics). Figure 3 summarizes 
basic steps of the latest version of the candidate math model search algorithm that is used to obtain the 
recommended math model. Different user choices (inputs) are listed on the left hand side of the flow chart 
that is depicted in Fig. 3. Each user choice influences the search process and the final term combination that 
the recommended math model will have. 

Two significant improvements of BALFIT’s candidate math model search algorithm were made in 2010. 
The first improvement is related to different search strategies that may be chosen during the regression model 
search. Now, an analyst may use forward selection or backward elimination as a search strategy . Table 4 
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below compares advantages and disadvantages of the two search strategy choices: 


Table 4: Search Strategy Options. 



ADVANTAGES 

DISADVANTAGES 

FORWARD 

• Constraints can be enforced simultaneously 

• Benefit of term combinations may be overlooked 

SELECTION 


• Only suited for automated search 

BACKWARD 

• Benefit of term combinations is recognized in every step 

• Constraints cannot be enforced simultaneously 

ELIMINATION 

• Suited for both automated and non— automated search 



The primary search strategy, i.e., forward selection , has the advantage that both threshold based search 
constraints (constraint 2 & 3 in Fig. 3) can be enforced simultaneously (i.e, in parallel). Forward selection , 
however, may overlook benefits of certain term combinations as the regression model is built by adding 
terms during the search. Forward selection is also only suited for an automated search. The initial number 
of required math model tests is simply too large for a visual inspection of different statistical quality metrics 
that may be used to interactively select terms of the regression model. 

The alternate search strategy, i.e., backward elimination , recognizes the benefit of math term combina- 
tions during every step of the search as the regression model is built by removing terms during the search. 
Backward elimination , however, cannot enforce the threshold based search constraints (constraint 2 & 3 in 
Fig. 3) simultaneously as a term selected for removal may not necessarily violate all constraints. Therefore, 
it was decided to implement the search constraints in “series” if backward elimination is selected as the 
search strategy. This implementation works in three steps. First, a regression model is tested using search 
constraint 2 (near-linear dependency test) with a temporarily elevated threshold of 50. This test has priority 
over search constraint 3 (test of the significance of a term) until search constraint 2 is no longer violated 
during the search process. Then, search constraint 3 takes over. Now, search constraint 3 has priority over 
search constraint 2. Finally, search constraint 2 takes over again using the thresholds of 5 (or 10) after search 
constraint 3 can no longer be violated during the search process. 

A second improvement was also implemented in the latest version of the candidate math model search 
algorithm. Previously, only the minimum of the search metric was used to identify the recommended math 
model after the completion of the candidate math model search. Now, it is also possible to use the 
maximum of the F- Value of Regression of the candidate math models for this purpose. Table 5 below lists 
the two identification metrics that an analyst may choose to identify the recommended math model after 
the completion of the candidate math model search. 

Table 5: Identification Metric Options. 



DESCRIPTION 

PRIMARY METRIC 

MINIMUM OF THE SEARCH METRIC 

ALTERNATE METRIC 

MAXIMUM OF THE F- VALUE OF REGRESSION 


The minimum of the search metric is still used as the default choice for the identification of the rec- 
ommended math model. The new identification metric, i.e., the maximum of the F- Value of Regression of 
the candidate math models, may also be selected. This alternate metric usually recommends a math model 
that has fewer terms than the math model that the primary identification metric suggests. A more detailed 
discussion of the new alternate identification metric is given in the appendix of the paper. 

Experience with different types of experimental data sets has shown that it is important for 
a user of BALFIT’s regression model optimization process to remember that different combinations 
of the search strategy ( forward selection -or- backward elimination ) and of the identification metric 
( minimum of the search metric -or- maximum of the F— Value of Regression ) may have to be tested in or- 
der to find a regression model that meets an analyst’s expectations. For the current study, however, it was 
decided to always use backward elimination and the minimum of the search metric for the optimization of a 
regression model as this combination lead to reasonable optimization results in all cases. 
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V. Analysis in Direct-Read Format 


During the first part of the study the calibration data of the MC60D balance was processed in direct- 
read format. Therefore, the balance loads were used in the following format: NF, PM, SF, YM, RM, and 
AF. The strain-gage outputs, on the other hand, remained unchanged. Input data files were prepared and 
calibration data points were processed using the three chosen load calculation methods. BALFIT’s candidate 
math model search algorithm was applied in each case in order to obtain an optimized regression model of the 
data that met strict statistical quality requirements. Afterwards, load residuals and corresponding standard 
deviations were computed at the data points (subset 1) and the confirmation points (subset 2). 

Table 6.1 and 6.2 below show the standard deviation of the load residuals of the calibration data points 
and the confirmation points for the three methods in both engineering units and percent of load capacity. 

Table 6.1: Standard deviation of calibration data points (subset 1) in direct-read format. 


METHOD 

NF, lbs 

PM , in-lbs 

SF, lbs 

Y M , in-lbs 

RM , in-lbs 

AF, lbs 

Iterative 1 
Iterative 2 
Non-Iterative 

2.50 (0.050%) 
diverg. 

2.46 (0.049%) 

9.61 (0.048%) 
diverg. 

9.62 (0.048%) 

2.59 (0.104%) 
diverg. 

2.58 (0.103%) 

11.56 (0.116%) 
diverg. 

11.87 (0.119%) 

7.27 (0.146%) 
diverg. 

7.27 (0.145%) 

0.43 (0.061%) 
diverg. 

0.43 (0.061%) 


Table 6.2: Standard deviation of confirmation points (subset 2) in direct-read format. 


METHOD 

NF, lbs 

PM , in-lbs 

SF, lbs 

Y M , in-lbs 

RM , in-lbs 

AF, lbs 

Iterative 1 
Iterative 2 
Non-Iterative 

2.46 (0.049%) 
diverg. 

2.45 (0.049%) 

9.92 (0.050%) 
diverg. 

9.91 (0.050%) 

2.65 (0.106%) 
diverg. 

2.60 (0.104%) 

12.14 (0.121%) 
diverg. 

12.55 (0.125%) 

7.44 (0.149%) 
diverg. 

7.55 (0.151%) 

0.45 (0.065%) 
diverg. 

0.45 (0.065%) 


It is observed that the standard deviations of Iterative Method 1 and the Non-Iterative Method for the 
points of subset 1 and subset 2 show very good agreement. We also see that Iterative Method 2 did not 
converge. This behavior is directly linked to the existence of the primary gage sensitivities of the balance 
calibration data. Table 6.3 below shows the primary gage sensitivities of the balance in direct-read format. 

Table 6.3: Primary gage sensitivities of balance calibration data in direct-read format. 


d NF 

dRi 


d PM 
dR 2 


d SF 
dR 3 


dYM 
d R 4 


d RM d AF 

d R§ d Rq 


lbs 



in — lbs 

l hV/v\ 



[ frV/V \ 


lbs 



in — lbs 

lhV/v\ 



[ vV/V \ 


in — lbs 



' lbs ' 

[ »V/V \ 



[hV/v\ 


SENSITIVITY 


not defined 


not defined 


not defined 


not defined 


3.7689 


0.8330 


We see that the sensitivities of the first four load/gage output combinations are not defined. This 
observation explains the divergence of the load iteration scheme as Iterative Method 2 implicitly assumes 
that all primary gage sensitivities of the balance gages of the chosen load format exist (see Ref. [3] for a 
detailed description of Iterative Method 2). 

VI. Analysis in Force Balance Format 

In the second part of the study the data of the MC60D was processed in force balance format as the 
balance gages behave like gages of a force balance. Therefore, the loads were converted to the following 
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format: N\, N 2 , Si, S 2 , RM, and AF. The transformation equations are defined as follows (from Ref. [6]): 


Ni = 

NF 

PM 

( 3 «) 

2 + 

P 

n 2 = 

NF 

PM 

(3b) 

2 

P 

Q, 

SF 

YM 

(3c) 

Ol — 

2 + 

q 


SF 

YM 

(3d) 

02 — 

2 

q 


where the distances p and q between the forward and aft gages of the normal and side force components are 
given as 

p = q = 6.0 [in] (4) 

The transformed calibration loads and the original gage outputs were processed using the three load 
calculation methods. Again, optimized regression models of the data were used for the analysis of the 
calibration data. Table 7.1 and 7.2 below show the standard deviation of the load residuals of the calibration 
data points (subset 1) and the confirmation points (subset 2) for the three methods in engineering units. 

Table 7.1: Standard deviation of calibration data points (subset 1) in force balance format. 


METHOD 

Ni, lbs 

N 2 , lbs 

Si, lbs 

S 2 , lbs 

RM , in-lbs 

AF, lbs 

Iterative 1 

1.66 

2.36 

2.41 

2.24 

7.17 

0.43 

Iterative 2 

1.66 

2.36 

2.41 

2.24 

7.17 

0.43 

Non-Iterative 

1.71 

2.31 

2.43 

2.28 

7.27 

0.43 


Table 7.2: Standard deviation of confirmation points (subset 2) in force balance format. 


METHOD 

Ni, lbs 

N 2 , lbs 

Si, lbs 

S 2 , lbs 

RM, in-lbs 

AF, lbs 

Iterative 1 

1.66 

2.41 

2.52 

2.33 

7.33 

0.45 

Iterative 2 

1.66 

2.41 

2.52 

2.33 

7.33 

0.45 

Non-Iterative 

1.72 

2.38 

2.54 

2.36 

7.55 

0.45 


This time, the primary gage sensitivities of all balance gages are defined (see Table 7.3 below). Therefore, 
Iterative Method 2 also converged and corresponding balance load residuals were computed. 

Table 7.3: Primary gage sensitivities of balance calibration data in force balance format. 


dNi 

dRi 


dN 2 

d IF 


d Si 

~dlh 


dS 2 

d R 4 


d RM d AF 

d R 5 d R 6 


lbs 


lbs 


lbs 


lbs 


in — lbs 


lbs 


pV/V 


pV/V 


pV/V 


pV/V 


pV/V 


pV/V 


SENSITIVITY 


2.6396 


2.1571 


2.1668 


2.1341 


3.7689 


0.8330 


Several observations can be made if results listed in Table 7.1 and 7.2 are compared: (1) all methods 
appear to have a similar load prediction accuracy as the standard deviations of the load residuals of both 
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subsets show very good agreement; (2) the standard deviations of both iterative methods are identical. The 
second observation is expected as both iterative methods, by design, are developed from the same set of 
regression coefficients that were used to fit the data. 

VII. Comparison of Results 

It would be interesting to directly compare results in force balance format (Table 7.1 and 7.2) with results 
in direct- read format (Table 6.1 and 6.2). This comparison would allow us to see if the load format selection 
influenced the load prediction accuracy. The comparison can be performed if the standard deviations reported 
in Table 7.1 and 7.2 are converted from force balance to direct-read format. Afterwards it is possible to 
express the converted values as a percentage of the balance load capacity which will simplify the comparison 
of the converted values with those values that are given in Table 6.1 and 6.2. 

The format conversion equations for the standard deviation can be understood by discussing a simple 
example. Let us assume that variables a and (3 are a set of two independent variables that describe the 
behavior of a dependent variable called 7. Then, we can write: 

7 = i{ol,P) (5a) 

We know from calculus that the individual standard deviations of the independent variables are related 
to the standard deviation of the dependent variable using the following relationship: 


cr(7) 




d 7 
d a 



d 7 

~dj 



(5b) 


We also know, after applying the classical strain-gage balance load transformation given in Ref. [6] to 
the balance loads, that the following relationship between the total normal force and its components is valid: 


NF = 7 (N u N 2 ) = Ni + N 2 


(6a) 


For the total pitching moment at the balance moment center the following relationship applies: 

PM = n(Nt,N 2 ) = (N x - N 2 )-p/ 2 ( 66 ) 

Now, after applying Eq. (56) to Eq. (6a), we get for the standard deviation of the total normal force 
the relationship: 

v(NF) « yj HV)] 2 + [a(N 2 )} 2 (7) 

Similarly, after applying Eq. (56) to Eq. (66), we get for the standard deviation of the total pitching 
moment the relationship: 

a(PM) « -|- • yj HN,)) 2 + [ a(N 2 )] 2 ( 8 ) 

Equivalent relationships can be derived for SF and Y M using the side force components Si and S 2 as 
independent variables. Now, after converting the standard deviations given in Table 7.1 and 7.2 from force 
balance to direct-read format, we get the following results in engineering units and in percent of the load 
capacity: 


Table 8.1: Converted standard deviation of calibration data points (subset 1). 


METHOD 

NF , lbs 

PM , in-lbs 

SF, lbs 

~Y M , in-lbs 

RM , in-lbs 

AF, lbs 

Iterative 1 
Iterative 2 
Non-Iterative 

2.88 (0.058%) 
2.88 (0.058%) 
2.87 (0.057%) 

8.65 (0.043%) 
8.65 (0.043%) 
8.62 (0.043%) 

3.29 (0.131%) 
3.29 (0.131%) 
3.33 (0.133%) 

9.87 (0.099%) 
9.87 (0.099%) 
9.99 (0.100%) 

7.17 (0.143%) 
7.17 (0.143%) 
7.27 (0.145%) 

0.43 (0.061%) 
0.43 (0.061%) 
0.43 (0.061%) 
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Table 8.2: Converted standard deviation of confirmation points (subset 2). 


METHOD 

N F , lbs 

PM , in-lbs 

SF , lbs 

Y M, in-lbs 

RM , in-lbs 

AF, lbs 

Iterative 1 
Iterative 2 
Non-Iterative 

2.93 (0.059%) 
2.93 (0.059%) 
2.93 (0.059%) 

8.78 (0.044%) 
8.78 (0.044%) 
8.80 (0.044%) 

3.43 (0.137%) 
3.43 (0.137%) 
3.47 (0.139%) 

10.28 (0.103%) 
10.28 (0.103%) 
10.40 (0.104%) 

7.33 (0.147%) 
7.33 (0.147%) 
7.55 (0.151%) 

0.45 (0.064%) 
0.45 (0.064%) 
0.45 (0.064%) 


Finally, after comparing values reported in Table 6.1 and 6.2 with corresponding converted values given 
in Table 8.1 and 8.2, we see that the standard deviations for the three tested methods show good agreement. 
The predictive capability of all three methods appears to be compatible keeping at the same time in mind 
that the regression models used by the tested load calculation methods were not identical. 

VIII. Conclusions 

Data from the machine calibration of a six-component force balance was used to investigate the accuracy 
of iterative and non-iterative strain-gage balance load calculation methods. Two iterative and one non- 
iterative method were used for the analysis of the calibration data. The balance loads were also processed in 
both the direct-read and force balance format. In addition, the latest improvements of a regression model 
optimization process were discussed in some detail as the optimization process was used to select regression 
models for the analysis of the balance calibration data. 

Standard deviations of the load residuals of calibration data points and regression model independent 
confirmation points were compared after the completion of the regression analysis of the calibration data. 
In addition, standard deviations obtained in force balance format were transformed to corresponding values 
in direct-read format so that the influence of the load format on the load prediction accuracy could be 
investigated. 

Overall, all three load calculation methods appear to have compatible load prediction accuracies. In 
addition, it was noticed that the second iterative method does only converge to a solution if the balance 
data is processed in force balance format. This observation can be explained by the fact that the iteration 
equation of the second iterative method implicitly assumes that the primary gage sensitivity of all gages 
exists. This assumption is violated whenever a force (or moment) balance data set is processed in direct- 
read format. The first iterative method, on the other hand, can process calibration data in both direct read 
and force balance format as the iteration equation of this method does not depend on the existence of the 
primary gage sensitivities. 
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Appendix: Identification of Recommended Math Model 


A regression model optimization process is being used at the Balance Calibration Laboratory at NASA 
Ames Research Center for the automated regression analysis of strain-gage balance calibration data and other 
multivariate experimental data sets (see Ref. [2], [3], and [4] for a detailed description of the optimization 
process that uses a candidate math model search algorithm). At the beginning of 2010 an important addition 
to the regression model optimization process was made. Now, the maximum of the F-Value of Regression 
of the candidate math models may be used as an alternate identification metric for the selection of the 
recommended math model after the completion of the candidate math model search. 

Key ideas of the candidate math model search algorithm need to be reviewed in order to better un- 
derstand the advantage of using the maximum of the F-Value of Regression as an alternate identification 
metric for the selection of the recommended math model. In principle, the candidate math model search 
algorithm minimizes a search metric during the search. This search metric is the standard deviation of the 
PRESS residuals of a regression model if no confirmation points are a part of the experimental data set (see 
Ref. [5] for a discussion of different search metric options). By design, the search metric is a monotonically 
decreasing function for most data sets if it is plotted versus the number of regression model terms. One of 
the candidate math models needs to be chosen as the recommended math model after the completion of the 
search. 

Different techniques were implemented over the years in order to select the recommended math model 
from the final set of candidate math models. Initially, the search algorithm applied a user selected “empirical” 
threshold in order to identify the recommended math model. This threshold was compared with the difference 
of the search metric value of two consecutive candidate math models. The recommended math model was 
picked as soon as the difference was less than the specified threshold (see Ref. [3], p. 3, for a description of 
this initial selection approach). This initial identification method is no longer used in the algorithm. 

A major revision of the search algorithm was made in 2007. At that time more advanced statistical 
metrics and several search constraints were introduced in the candidate math model search algorithm. This 
improved version of the algorithm no longer required a user input for the selection of the recommended math 
model. It was simply chosen to be the candidate math model that had the smallest search metric value 
while satisfying all search constraints. This identification metric, i.e. , the minimum of the search metric , is 
still being used as the default choice in the current version of the algorithm. It is the primary identification 
metric for the selction of the recommended math model. 

Studies were performed at the beginning of 2010 that compared the F-Value of Regression of the 
candidate math models with the corresponding plot of the search metric. During these studies an interesting 
connection between the F-Value of Regression and the search metric was observed: 

The F— Value of Regression has a maximum for the candidate math 
model that gives the “best return on investment” assuming that the 
number of regression model terms is considered to be the “investment” 
and the magnitude of the search metric is considered to be the “return.” 

It was also observed that the math model identified by the maximum of the F-Value of Regression has 
fewer terms than the math model that would be chosen by using the primary identification metric discussed 
in a previous paragraph. Therefore, it was decided to introduce the maximum of the F-Value of Regression 
of the candidate math models as an alternate identification metric for the selection of the recommended 
math model as this metric appears to pick a recommended math model that uses the smallest possible set 
of significant regression model terms. 

It still needs to be shown mathematically that the F-Value of Regression of the candidate math models 
has a maximum if it is plotted versus the number of regression model terms of the candidate math models. 
The proof starts with the definition of the F-Value of Regression of a regression model of some experimental 
data. It is defined as follows (from Ref. [7], Eq. (2.21)): 

SS R /k = SS R _ n-k-1 

SSe / (n — k — 1) SSe k 

where SS R is the sum of squares due to regression, SSe is the sum of squares due to residual, n is the 
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number of data points (observations), and k is the number of terms of the regression model (not counting 
the intercept). We also know that the total sum of squares, i.e., S yy , is the sum of (i) the sum of squares due 
to regression and (ii) the sum of squares due to residual (see Ref. [7], Eq. (2.20)). Therefore, we can write: 


Syy = SS R + SSe (10a) 

or, after solving Eq. (10a) for SSr, we get 

SS R = Sy y - SS E (10 b) 


Now, after using the right hand side of Eq. (106) to replace SSr in Eq. (9), we get: 


F 0 


' Syy - SSe ' 


n — k — 1 

[ SSe \ 


k 


( 11 ) 


We also know from the literature that the total sum of squares (S yv ) is only a function of (i) the 
dependent variables (responses) and (ii) the number of data points. It is defined as (see Ref. [7], p. 29): 


S„ 



(12a) 


where iji is a dependent variable value (response value) at data point i. Similarly, we know that the sum of 
squares due to residual (SSe) is defined as (see Ref. [7], p. 26): 


ss E = Y (y* - M 2 ( 126 ) 

i—1 


where y t is the fitted dependent variable value (response value) at data point i. We know from Eq. (11) that 
the F-Value of Regression of the candidate math models is closely linked to the search metric value because 
the search metric is the standard deviation of the PRESS residuals of the experimental data set, i.e., a value 
that closely follows the behavior of the sum of squares due to residual of the fit. Therefore, it is sufficient 
to look at the behavior of the sum of squares due to residual (SSe) of the candidate math models instead 
of looking at the search metric itself. 

The proof has to examine the different terms that are used in Eq. (11) to compute the F-Value of 
Regression. First, the total sum of squares (S yy ) is investigated. From Eq. (10a) we see that S yy is 
independent of the regression model of the data. It is a constant for the experimental data set. This 
conclusion can be summarized as follows: 


Conclusion 1 : S yy => constant for experimental data set 

The situation is different for the sum of squares due to residual. This value decreases initially and 
approaches an asymptote as it closely follows the behavior of the search metric that is used during the 
candidate math model search. Therefore, we can summarize this conclusion using the following statement: 

Conclusion 2 : SSe => decreases and approaches an asymptote as k increases 

Now, knowing that S yy is a constant (Conclusion 1) and that SSe decreases and approaches an asymp- 
tote as the number of regression model terms increases (Conclusion 2), we can make the following statement 
about the difference of the two values: 


Conclusion 3 : S yy — SSe => increases and approaches an asymptote as k increases 

Similarly, after computing the inverse of SSe and after considering Conclusion 2, we can say: 


Conclusion 4 : 


1 

~SS^ 


increases and approaches an asymptote as k increases 
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Now, after combining Conclusions 3 and 4, we can make the following statement about the first factor 
on the right hand side of Eq. (11): 


Conclusion 5 : 


S VV - SS E 


SSe 


increases and approaches an asymptote as k increases 


Similarly, examining the second factor on the right hand side of Eq. (11), we can make the following 
two statements: 

Conclusion 6 : n — k — 1 => decreases as k increases 


Conclusion 7 : —— 

k 


decreases as k increases 


Therefore, after combining Conclusions 6 and 7, we can make the following statement about the second 
factor on the right hand side of Eq. (11): 


Conclusion 8 : 


n — k — 1 


decreases as k increases 


We see, after applying Conclusion 5 and Conclusion 8 to the definition of the F-Value of Regression 
given in Eq. (11), that the first factor (S yy — SSe)/SSe increases initially as k grows. At some point, 
however, the first factor approaches an asymptote. The second factor (n — k — 1 )/k, on the other hand, 
decreases monotonically as k grows. Therefore, the F-Value of Regression must have a maximum if plotted 
as a function of the number of terms of the candidate math models. This statement can be summarized as 
follows: 


Conclusion 9 : F 0 => must have a maximum if plotted versus number of terms 

Finally, the above conclusions can be summarized as follows: (i) The regression model of the 

maximum of the F-Value of Regression may be considered to be the regression model that gives the “best 
return on investment” assuming that the number of regression coefficients k is considered to be the “in- 
vestment” and the sum of squares of residual SSe is considered to be the “return.” (ii) Therefore, because 
SSe closely follows the search metric value of the candidate math models, the candidate math model of the 
maximum of the F-Value of Regression should be chosen as the recommended math model. 

Calibration data of the MC60D balance may be used to illustrate the connection between the search 
metric, the maximum of the F-Value of Regression of the candidate math models, and the recommended 
math model. Therefore, it was decided to use the non-iterative method in order to obtain a regression 
model of the axial force as a function of the six gage outputs of the MC60D balance. The optimization of 
the regression model of the axial force was performed using backward elimination as the search strategy and 
using the maximum of the F-Value of Regression to select the recommended math model. 

Figure 4a shows the search metric of the 28 candidate math models that were identified during the 
search for an optimized regression model of the axial force. A total number of 279 math models were tested 
during the search in order to obtain the 28 candidate math models that are shown in Fig. 4a. Only 23 of the 
28 candidate math models fulfilled search constraints 2 & 3 (these models are marked using enlarged and 
boldface symbols in Fig. 4a). Figure 4b shows the F-Value of Regression of all candidate math models. It 
can be seen that the F-Value of Regression reaches a maximum for the candidate math model that appears 
to use a minimum number of significant terms (the blue symbol marks the maximum). This candidate math 
model should be selected as the recommended math model. 
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ITERATION EQUATION TYPE 


INITIAL GUESS 


FIRST SOLUTION 


F e = 


C x AR 


c" c 2 


H 


£-1 


Iterative Method 1 (see Ref. [1], p. 19, Eq. (3.3.7)) 


H„ = 


0---0' 

0---0 

0---0. 


F, = C x AR 


F ,= 


B x AR - Bi B 2 F £ _ 1 B x C 2 


H t 


F„ = 


Iterative Method 2 (see Ref. [3], p. 11, Eq. (28)) 


H n = 


0---0' 

0---0 

0---0. 

0---0 

0---0 


F 1 = B x AR 


Fig. 1 Definition of iteration equations used by Iterative Method 1 and 2 (see also Ref. [3]). 



Fig. 2 NASA’s MC60D strain-gage balance (photo courtesy of Triumph/Force Measurement Systems). 


14 

American Institute of Aeronautics and Astronautics 




Fig. 3 Candidate math model search algorithm of 2010. 
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Fig. 4b F-Value of Regression versus number of math model terms for candidate math models of axial force, 
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